In-water inspection is an essential task in the general maintenance and damage assessment of underwater structures. For example, the inspection of a ship hull always has formed the principal component of a periodic maintenance routine. More recently, ship hull inspections have become extremely important in respect to homeland security in view of the threat that ships entering ports and harbors for commerce may serve as carriers of nuclear weapons, explosives, deadly chemicals and other hazardous materials. To combat the foregoing clear threat, the deployment of existing, and development of new, remote detection technologies have become a national priority.
Like a periodic maintenance routine, the inspection of ship hulls, bridge pilings, dams and off-shore oil structures or pipelines can use the participation by skilled divers. Yet, the process of underwater search and inspection in the context of security and risk containment has proven too dangerous for direct human involvement. In particular, the potential presence of hazardous and deadly materials has proven human involvement to be unsafe and requires the use of submersible robotics platforms to avoid risking human lives. As a result, in general it is expected that the deployment of unmanned underwater vehicles, when highly automated, will provide a more effective and efficient solution to the problem of underwater inspection.
Current generation submersible platforms, such as autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs), enjoy distinct operational advantages over the use of other submersible platforms. Exemplary operational advantages include real-time video and data transmission to the operator station which can enable the operator to revise the mission “on the fly” or take assume manual operation of the vehicle when necessary. Additional advantages include maneuverability and a division of labor by automating many low-level tasks as in precise navigation and the construction of a composite map of the target structure, while the operator can concentrate on high-level critical components as in target and object recognition, threat assessment, and the like.
In recent years, each of automated video-based surveillance, surveying and mapping have become recognized as the principal important capabilities of the AUV and ROV, particularly in respect to ocean exploration, and seafloor and benthic studies. The goal is to process the video imagery online, as acquired, to determine the position of the AUV or ROV relative to the target of interest, and to automatically navigate under computer control to inspect and map the various parts of the target structure. While the absence of natural lighting at depth can pose serious challenges in deep sea operations, other serious complexities in the computer processing and analysis of the video can arise in shallow waters in the course of performing automated inspection. Such complexities can include those which arise due to non-uniform moving shadows cast by surface waves, floating suspended particles, and the like.
Video-based servo and mapping represent two popular applications of vision techniques applied underwater. The fundamental problem is to estimate the frame-to-frame motions of the camera from the corresponding pair of images. Moving shadows represent one of the major complexities in the processing of underwater video. In deep sea, these are induced by the motion of artificial sources with limited power. For shallow water inspection, one is primary concerned with disturbances near the sea surface.
In shallow waters, disturbances arise from the surface waves that cast shadows on the structure to be imaged. In some cases, for instance where the target surfaces have weak texture, these shadow artifacts can dominate the image variations induced by camera movements which comprise the primary visual cues for determining the motion information. In addition, other complexities arise from the movement of floating suspended particles and water bubbles that are commonly present in shallow waters.
Estimation of the various degrees of freedom in the motion of the vehicle can be important for both position control and target mapping. As stated, the instantaneous pose including position and orientation of the submersible platform relative to the target is sought, rather than its absolute three-dimensional position, as the most relevant information. Most small-size ROVs include four thrusters—two aft, one lateral and one vertical. The thrusters can be applied for X-Y-Z translations and heading change. A video camera can be installed at the front of the ROV and can be aimed anywhere including the forward and downward directions.
The ideal mode for visual servo, when a submersible vehicle navigates along the sea floor, is the down-look configuration. FIGS. 1A through 1C, taken together, are a pictorial illustration of an ROV 110 configured with a single optical camera 120. Referring first to FIG. 1A, the camera 120 can be positioned in a down-look orientation for sea-floor mapping. The down-look configuration can be preferred because four of the six degrees of freedom in the motion of the vehicle 110, controllable through the proper signals to the four thrusters of most common ROVs, are the same four degrees of freedom that can be estimated most reliably from the video frames of the sea floor.
Notably, the skilled artisan will recognize that one can claim that the controllable system states are all observable. Yet it will also be recognized that the same may not be assumed about uncontrollable states, namely pitch and roll motions. While the pitch and roll states theoretically can be determined from video, the estimation is seldom robust and accurate particularly where the target scene (seafloor) is relatively flat. That is, where the topographical variations are small compared to the distance to the sea floor, it can be difficult to estimate the pitch and roll states of the visual servo. Accordingly, the most ideal scenario to maintain positioning accuracy by visual servo is to navigate with no, or very little, pitch and roll motion.
To observe and estimate these other motion components, inexpensive angle sensors are often sufficient. In this case, the video is rectified to correct for (stabilized with respect to) pitch and roll motions, before processing to estimate the other four degrees of freedom, providing all the necessary information for positioning and producing a mosaic. By comparison to seafloor mapping, for hull inspection, the ROV 110 can traverse the vertical sides of the ship, maintaining a constant distance and orientation relative to the ship. In one scenario, illustrated in FIG. 1B, an extra camera 130 is installed in a side-look arrangement while the vehicle 110 moves forward or backward along the hull. Alternatively, as shown in FIG. 1C, the existing camera 120 may be pointed in forward-look configuration, while the vehicle 110 moves sideways (left and right) to map the ship. In either configuration, the change in the heading of the ROV 110 corresponds to the pan motion of the camera 120, which often cannot be estimated with good accuracy from video when coupled with side-to-side translation. Unfortunately, the heading change cannot be reliably measured from typically compasses due to magnetic masking near the ship.